HITHER GATE MUSIC

An Introduction to Sound Design
~ From Real-life to Imaginary Landscapes ~

by Archer Endrich

First a Recording

Everyone has been given a jar cap with a safety button. Please click the button (everyone at once) when my hand comes down: fast for ca 2 sec, slow for ca 2 sec, very fast for ca 2 seconds and stop when my hand sweeps horizontally. First practice, then record. Thanks. We'll come back to that later.

Sound comes of age

Looking back over the previous 100 years, one might dub the 20^th century the 'century of sound'. The gongs of the Javenese Gamelan heard at the Paris Exhibition in 1890 inspired composers such as Debussy and Poulenc. The Italian Futurists began consciously to absorb and work with the sounds of the increasingly mechanical world around them, picking up initially on their dynamic power and energy. With the beginning of radio, the sound studio was born and it wasn't long until composers wanted to get in and work with the equipment. Pioneers such as Edgard Varèse, Karlheinz Stockausen and Luciano Berio were among them. Soon there was sound for film and the whole business of creating a soundtrack, an acoustic environment for the film began to take shape. The creative tool moved from tape recorder to computer, and in the past 60 years many thousands of musical compositions have been created that involve sounds in one way or another. It is one of the most important and vigorous areas of a hugely broad and complex field: MUSIC!

This presentation can therefore be called an "overview of working with sound". The emphasis is on sound design for films, as a useful point of reference. Sound design creates a completely believable acoustic environment for visual images (real or CGI!) OR a completely imagined and abstract acoustic environment – and everything inbetween. CGI is the computer graphics imaging that we are familiar with in for example the Pixar films such as Finding Nemo, Monsters Inc, Toy Story, Flushed Away etc. Although animations, the realistic sound tracks add considerably to the effectiveness of these films.

A top reference work: Sound Design - The Expressive Power of Music, Voice, and Sound Effects in Cinema by David Sonnenschein. Micheal Wiese Productions. Studio City, CA. 2001.

Foley

First, let's try to think of some real-life sounds: footsteps, doors opening and closing or sliding, horns honking, equipment hums, people talking in groups, the wind sighing or making the trees moan, waves lapping or crashing ...

• 'Foley' is a whole branch of sound design for radio and films.
• It recreates real-life sounds in very clever ways in studio recording conditions. The purpose is to get a 'clean' recording that will put the right sound in the right place without unwanted sonic additions in the background (car, voice sparrow, plane, wind ...). Another reason is so that the music and sound effects need to be completely separate from the dialogue track to enable the creation of international dubbed versions.
• For example, footsteps. Foley artists will have many different surfaces in their studio on which to walk and many different types of shoe to wear (they seldom fit the wearer!). The foley performer walks on the appropriate surface with the appropriate footgear, timing his actions to those of the on-screen character.
• Foley is a highly skilled activity and its performers are rightly called 'artists'. The whole business of Foley calls for extreme inventiveness, as is well-illustrated by early radio and cartoons.

ADR

As mentioned above, the Foley effects are almost always not the sounds heard on location, and music + effects and dialogue are on separate tracks. ADR, which means Automated Dialogue Replacement, is how this is done with the spoken word.

The actors watch themselves on-screen over and over in the studio and practice speaking the dialogue with accurate lip sync and appropriate emotional content. This is challenging work, the more so with CGI-based films. The result is a sound track that is clean, able to be enhanced as needed, and transferable.

Foley-illusion

Foley-illusion comes in two ways or things aren't always what they seem.

1. An existing Foley recording needs to be enhanced.
   • A person walks from inside a lakeside chalet, down a gravel path that goes to a short wooden quay, gets into a small power boat and sets off across the lake into the gathering dusk.
   • Dramatically, we may want to focus on the footsteps with no other music or voiceover. Perhaps it was a big or fateful decision to go and we want to emphasise that decision. But getting just the right sound isn't so easy.
   • Footsteps inside the chalet: wooden floors and walls, so they should be loud, sharp and a bit echoey. We may therefore want to enhance the loudness (amplitude gain), the sharpness (tighten the attack envelope of each step), and make them a bit more echoey (add reverb).
   • Footsteps down the gravel path: downhill, so the footsteps will dig into the gravel. We may therefore want to do something to add scrunch to each step, plus some outdoor ambience such as the dusk call of a blackbird, the sound of a nightjar, a touch of wind in the trees.
   • Footsteps on the wooden quay: these will have a different sort of resonance, because hollow beneath. We may therefore want to alter the resonance accordingly and perhaps add the sound of water lapping at the shoreline.
   • Stepping into the boat: this is yet another surface and type of resonance. The boat starts up and sets off. A tricky resonance task here. The computer can gradually lower the volume to make the sound of the boat quieter as it gets further away. The eerie sound of a loon echoing across the lake might be appropriate. Music might start now to lead to and set up the next scene.
   • So you see how much there is to do to realise the sonic dimension of this short yet crucial scene, how many decisions need to be made, the technical challenges to get the sound 'just right', and what opportunities there are to interpret the scene dramatically through sound. When it's right we instantly understand the scene and probably don't even notice the sounds, much less think about what is real and what might be 'enhanced'.
2. Sometimes the required sound is best made by something else entirely.
   • My favourite example of this is a scene in which someone is walking across a frozen Arctic landscape. Perhaps the sound of the snow on the day just didn't give enough of an impression of coldness, perhaps the wind was too strong. The real-life recording actually had on it "Where's my cup of tea?" It just wouldn't do at all. So the footsteps were made by scrunching two pieces of polystyrene together. (NB: it was a very inventive and skillful use of Foley!).
   • Both The Red Planet and Iron Man had metallic men in them that weren't 'real. The first had a robot (that became deranged) and the second a man in a high-tech metallic suit. The whirring, buzzing, clashing and crashing had to be made by other objects, and then no doubt appropriately enhanced.
   Another example is the creaking of the Titanic in Titanic as it gradually sank. The sound designers made over 300 creaking sounds and arranged them on a scale of 'creakiness', from least to most. This series of sounds positioned the viewer/listener as the situation of the vessel worsened.

Composite sounds

Sounds to accompany action have to have just the right dramatic 'feel', and a simple or even enhanced recording may not be enough. One solution to this is to combine sounds to get a composite sound. Here's another example from Titanic. The doors between compartments need to be closed to hold back the water for as long as possible. What kind of door would sound believable and carry lots of dramatic overtones? The solution was to record a big metallic jail door clanking shut and combine it with the sound of a large empty oil drum being cast down onto the ground.

There are many sound 'tricks' that can be done with different sounds. For example, suppose we wanted to have an ad in which the sound of a tractor purred gently like a cat? Sound of a tractor, Sound of a cat purring. What we do is substitute the sound of the tractor for the sound of the cat, but within the amplitude envelope of the cat, i.e., the rise and fall of volume as it purrs. There is more than one way to do this: either amplitude shape the sound of the tractor with the envelope of the cat, or do a full cross-synthesis, actually replacing the sound of the cat with the sound of the car. We have used the second method to produce this: purring tractor engine. And here is a cat purring with the sound of a Jaguar XIS. Source Jaguar XIS.

Playing with the voice

These examples of vocal transformation gives some idea of how finely one can tune the sonic image. Here is the unaltered vocal sound: count.wav.

Most of these belong to a fairly mild 'roughen the voice' category, plus a few 'stranger' alterations (last 4). Here they are one after another, with descriptions below. We can play to any of these individually as well.

1. BLUR with 20 windows keeps the text clear, but adds a tinny sheen to it:
   countblur20.wav
2. BLUR with 70 windows slurs the speech a little and makes the tinny sheen more prominent:
   countblur70.wav
3. DISTORT & PITCHWARP (slightly) 0.2 octave movement puts a slight quaver into the voice that makes it sound a little uncertain:
   countpw_2.wav
4. DISTORT REPLACE with cycles = 2 and now the voice sounds really old and quavery:
   countdrp12.wav
5. DISTORT INTERPOLATE with 2 cycles elongates the voice with a rough tone in a sinster way:
   countdinterp.wav
6. DISTORT AVERAGE with 2 cycles and a wavelength of 0.9 and the voice goes high and thin, with intermittent rasps.
   countdavrg.wav
7. DISTORT MULTIPLY with 2 cycles and the voice becomes higher / younger, but the distortion makes it sound perhaps a bit cheeky as well?
   countdmult.wav
8. DISTORT ENVELOPE with a rising attack envelope set to 2 cycles and there is a slightly fuller tone:
   countdenv.wav
9. DISTORT REPEAT - 2 repeats of 2 cycles at a time makes the voice much deeper and rougher, a tough hombre:
   countdrpt2-2.wav
10. DISTORT REVERSE with groups of 50 cycles turned round and fragments of voice are turned around backwards.
    countdrev50.wav
11. DISTORT REVERSE with groups of 500 cycles turned round and it sounds like a different language!
    countdrev500.wav
12. BLUR CHORUS with amplitudes spread by (only) 100 and frequencies spread by (only) 1.2 and we hear a flurry of voices (a chorusing effect):
    countchorus.wav
13. BLUR BLUR (with 70 windows) applied to a voice tuned to an A-minor chord produces a rather synthetic sounding voice:
    counttuneAminb170.wav

Imaginary sonic landscapes

When one starts tweaking sounds as above, or piling transformation on transformation with hundreds of processing options, or making sounds entirely from numbers, new sound worlds emerge. We can no longer identify the source of the sounds or associate them with known objects, scenes or even emotions. We enter a world of imaginary sonic landscapes: variously called 'electronic music', 'musique concrète', 'electroacoustic music', 'acousmatic music', or just 'sound music'.

Sound music has been vigorously pursued and evolved during the 20^th century. It began with the Italian Futurists, inspired and thrilled by the sounds and energies of the industrial world. It continued with that great pioneer Edgard Varèse, of whom Frank Zappa said "Anyone who looks like a mad-scientist must be OK." It drew upon all the other amazing stylistic innovations of the century. Using early radio equipment, tape recorders, and then computer sound generation software, computer sound transformation software, and computer-based algorithmic composition, the field has advanced immeasurably, and many thousands of compositions of this kind have been created during the past 60 years.

Films increasingly draw upon these developments to get the sounds they need: for example to create 'atmospheres' for sci-fi films, or to put in a special evocative sound to set a scene or imply a subtext. The identity of the source sound is usually deliberately lost so that the new sound doesn't have any unwanted associations. Sound treatment has turned the source sound into something else, fit for purpose.

Our final section illustrates this by playing a complex sound and showing in part how it was achieved. Here is the final sound: : the imaginary image is that of 'comets' with long descending tails filled with harmonies and dispersing in clusters of incandescent sparkles. This sound comes from my soundscape composition Crossing the Dark Rift. It is at a part where Quetzalcoatl, the 'feathered serpent' begins his descent from among the stars.

• Source sound – single stroke on a Tibetan singing bowl: bdtg5dt.wav (4.9 sec), DOVETAILED to give a smooth start and finish.
• Fuzzy comet sound – the source sound is first made into a 4-note chord and then transformed with a type of 'pluck' algorithm to turn it into a fuzzy ball: noisefade2secdt.wav. 3 of these overlap.
• A musical chord – the original sound is transposed several times maintaining the same length, the transpositions layered by mixing , and then the new sound made to glissando downwards: tz5dwnchorddtstrd2tvd3.wav. 3 of these overlap.
• Bell-like sparkles – First we lighten the tone with a HI-PASS FILTER, i.e., let through the upper frequencies and cut out the lower ones. This creates baegcdthipc.wav. Then a cluster texture is shaped with an increase in density and amplitude towards the centre, then slowing again. Result:clustertz1s8g.wav. This also glissandos downwards: clustertz1s8gtvd3.wav. There are 3 of these (each different).
• Mix – all of the above sound files are mixed so that the fuzzy balls, glissandoing chords and glissandoing sparkles overlap to form a single complex sonic event – which may or may not conjure up in your imaginations the images I've used above to describe the sounds. Result: tz5s8mix.wav.

A recording like ours transformed

Here are some changes to a recording of just 2 button-caps: caps.wav. What happens is somewhat clearer than the considerably more complex click recording made at the meeting.

1. Transpose – lower 3 octaves (36 semitones) capsd36.wav
2. Rerhythm – closing gaps between the clicks capsrr.wav
3. Duplicate – each grain is repeated 5 times capsrrdupl5.wav
4. Blur x 20 – smooth over 30 analysis windows at a time capsrrdupl5bl20.wav
5. Stretch x 4 – stretch out 4 times capsrrdupl5x4.wav
6. Tune – to a C majorish chord capsrrdupl5x4tune.wav
7. Filterbank – tune with a changing-chords filter bank capsrrdupl5fb1.wav
8. Subharmonic – subharmonic filter capsrrdupl5x4subh.wav

Hands-on experiments

To complete the evening, I would like to invite you make your own transformations to the sound we recorded at the beginning of this session. To speed things along, I have pre-prepared the means of doing the transformations on the list to speed things along. See the list of options provided: Jan20list.html.

Last updated: 21 January 2011
Archer Endrich: archerhgm@talktalk.net